Reliability in Automated Aids for Unmanned Aerial Vehicle Flight Control: Evaluating a Model of Automation Dependence in High Workload

Twenty-four participants flew a simulated unmanned aerial vehicle (UAV) through ten mission legs while searching for targets of opportunity and monitoring system parameters. Participants were assisted by automation which provided auditory alerts in response to system failures (SF). The auto-alerts were either 80% reliable or 60% reliable; the latter condition resulted in either a 3:1 ratio of false alarms to misses, or vice versa. Results indicated that the 80% reliable automation exceeded baseline (no automation) performance in the target search task. The two 60% reliable conditions did not provide any benefits to performance; both false alarms and misses hurt performance in the automated task and concurrent tasks of target monitoring. The current data, and those from a prior study with different reliability values, are evaluated against a computational model of automation compliance and reliance, and it is observed that these two effects are linearly related to automation false alarm and hit rate, and are relatively independent of each other. Implications for this study suggest that automated aids must be fairly reliable to provide global benefits, and data regarding the relative costs of misses versus false alarms on performance were equivocal, suggesting qualitatively different sorts of effects of reliance and compliance.